1. Technological Field
The present disclosure relates to a method, apparatus, and computer-readable storage medium that remove flickering in video images of objects illuminated by ambient lighting and screen light.
2. Description of the Related Art
In order to have a meeting involving participants not located in the same area, a number of technological systems are available. These systems may include video conferencing, web conferencing, and audio conferencing.
A realistic substitute for real meetings is high-end video conferencing systems. Conventional video conferencing systems comprise a number of endpoints communicating real-time video, audio, and/or data streams over WAN, LAN, and/or circuit switched networks. The endpoints include one or more monitors, cameras, microphones, and/or data capture devices and a codec, which encodes and decodes outgoing and incoming streams, respectively.
Video conference systems are used throughout the business community for point to point audio and visual communication between individuals. The users of video conferencing systems may sit in workplace environments such as a personal office or a cubical, small or large meeting rooms, board rooms, or the like. Video conference applications installed on multi purpose computers have contributed to spreading the use of personal video conferencing even more.
FIG. 1 illustrates a situation in which an object (i.e., a person) captured by a camera sits close to the display screen 100. As screens become bigger and brighter, the screen 100 will illuminate the object which can lead to artifacts such as color and intensity variations on objects. The artifacts can visually manifest themselves in flickering or color variations that are modulated by the contents seen on the screen.
The flickering is caused by the exposure time (i.e., integration time) of the camera 110 to record each individual frame in a video picture, combined with the frequencies of the ambient lighting 120 and the refresh rate/frequency of the screen 100 illuminating objects captured by the camera 110. A screen 100 contains thousands of illuminative pixels in array. Visual content is displayed by successively applying voltage on each pixel in a pixel line from left to right for each line from the top of the screen 100 and downwards. The refresh rate of the screen 100 is therefore the frequency of illuminating the pixels in the whole screen 100 from top left to bottom right.
Depending on the refresh rate of the screen 100 in question, the refresh rate of the screen 100 may cause its own flickering even if the camera 110 is optimized for the normal ambient environment. Furthermore, with the advent of larger and high intensity emissive displays, indirect illumination caused by such emitters may cause flickering in all or portions of the image seen by the camera 110.
It may be desirable for the integration time to be as long as possible i.e., maximally responding to the frame rate (1/framerate) to maximize signal to noise (S/N) ratio in a resulting footage. However, due to line frequency of 50 Hz (60 Hz in the United States, 400 Hz in airplanes, for example), lamps operating at mains Alternating Current (AC) exhibit intensity variations at double the line frequency. To avoid flickering (i.e., phantom intensity variations over time, which may look like flickering or line rolling), an approach is to select an appropriate integration time for the environment with the aim of overcoming the problem during capture. The relationship that ensures cameras do not beat against a global cyclical stimuli f is:Integration time=n/(2f), where n=0, 1, 2, . . .
The result of adjusting the integration time according to this equation is that the camera 110 will capture integer numbers of half light wavelengths, thereby avoiding variations of light intensity among captured frames.
As already mentioned, flickering is not only caused by ambient light. In situations when captured objects are positioned close to relatively big screens, as discussed above, the illumination contribution from the screen 100 may be considerable, and if the integration time is adapted to ambient light only, flickering may occur due to the refresh rate of the screen 100.
FIG. 2 is a graph that shows the effect of a scene optimized for 50 Hz ambient at 10 ms integration time, with the intensity variations of various common screen refresh rates/frequencies when lit 100% by single sinusoidal intensity source. In FIG. 2, the X-axis represents complete 10 ms integration windows. As can be seen in FIG. 2, the intensity variations increase as the refresh rate moves away from 50 Hz. This is experienced as flickering by viewers watching the video images captured by the camera 110.